Applications such as bullet time and free viewpoint have been known from the past. In these applications, it is necessary to infer a photographing position and a photographing attitude of a captured image that is obtained by photographing a subject. In free viewpoint, for example, a photographing position and a photographing attitude of each of a plurality of captured images that are obtained by photographing a subject are inferred, and a captured image that is captured at an arbitrary virtual viewpoint position is generated through an interpolation process using the inference result and the captured images.
As such a technique of inferring a photographing position and a photographing attitude of a captured image, there is a technique of only using images, i.e., an image-based technique (e.g., refer to Non-Patent Literature 1). In a case in which a photographing position and a photographing attitude of a captured image to be processed in a 3-dimensional space are inferred using this technique, two captured images, of which photographing positions and photographing attitudes are known and the photographing positions in a 3-dimensional space are different from that of the captured image to be processed, are necessary.
That is, if the two captured images of which the photographing positions and the photographing attitudes in the 3-dimensional space are known are called reference captured images, positions of subject portions in the 3-dimensional space that correspond to feature points detected in the two reference captured images, in which the points correspond to each other in the images, can be calculated since the photographing positions and the photographing attitudes of the reference captured images are known. Thus, a photographing position and a photographing attitude of a captured image to be processed in the 3-dimensional space can be inferred on the basis of the calculated positions of the subject portions in the 3-dimensional space and positions of feature points of the captured image to be processed that correspond to the subject portions.